Electronic device and control method thereof for generating edited VR content

ABSTRACT

An electronic device includes a control unit to control so that cyclical scroll display of a same content is performed on the second screen, the cyclical scroll display involving, in accordance with a first operation, scrolling the VR content being displayed by flat display in a first direction without scrolling the indicator and sequentially displaying, in the first direction from an end in a second direction in the rectangular region, an image region corresponding to a scroll amount in the first direction among the VR content, and that the cyclical scroll display is not performed on the first screen even when the first operation is performed; and a generating unit to generate an edited VR content including a second video range that is narrower than the first video range among the VR content on the basis of a region indicated by the indicator.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an electronic device and a controlmethod thereof.

Description of the Related Art

In recent years, image capturing apparatuses capable of photographing aVR (Virtual Reality) image containing a video with a wider range than ahuman viewing angle such as an omnidirectional image or a fullycelestial image are becoming popular. On the other hand, an image formatspecialized for forward 180 degrees has been introduced, and demands forclipping a 180-degree image from a VR image are growing.

Japanese Patent Application Laid-open No. 2016-123127 proposes a methodof extracting a partial region on which a touch operation has beenperformed of a VR image, performing distortion correction, anddisplaying the corrected partial region in superposition on a screen.

Japanese Patent Application Laid-open No. 2014-165523 proposes a methodof designating a setting of a clipping region of an image by eitherchanging a size of a clipping frame or by changing a magnification ofthe image while keeping the clipping frame fixed.

When clipping a 180-degree clipping region from a VR image, a user canpanoramically view the entire VR image by performing flat display inwhich a 360-degree valid video range is fitted into a rectangle.However, in case of designating a 180-degree clipping region in thestate of flat display, there is a problem in that designating an endregion on the flat display as a clipping center is difficult.

SUMMARY OF THE INVENTION

In consideration of the above, an object of the present invention is toprovide an electronic device and a control method thereof which enable,in case of clipping a VR image with a narrower range from the VR imageon flat display, a region near an end of a screen of the flat display tobe designated as a clipping center.

In order to solve the problem described above, an electronic device ofthe present invention includes at least one memory and at least oneprocessor which function as: a display controlling unit configured tocontrol so that flat display involving fitting a first video range of aVR content into a rectangle is performed on a first screen and anindicator indicating a region to be clipped from the VR content isdisplayed on the VR content being displayed by flat display in arectangular region on a second screen; a switching unit configured toswitch the first screen to the second screen in accordance with asetting instruction of a clipping range from a user; a control unitconfigured to control so that cyclical scroll display of a same contentis performed on the second screen, the cyclical scroll displayinvolving, in accordance with a first operation, scrolling the VRcontent being displayed by flat display in a first direction withoutscrolling the indicator and, in accordance with the scrolling of the VRcontent, sequentially displaying, in the first direction from an end ina second direction that is an opposite direction to the first directionin the rectangular region, an image region corresponding to a scrollamount in the first direction among the VR content, and that thecyclical scroll display is not performed on the first screen even whenthe first operation is performed; and a generating unit configured togenerate an edited VR content including a second video range that isnarrower than the first video range among the VR content on the basis ofa region indicated by the indicator on the second screen.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a display control apparatus;

FIG. 2 is an external view of the display control apparatus:

FIG. 3 is a flow chart showing a clipping process from a VR image to a180-degree image;

FIG. 4 is a flow chart showing a clipping direction setting process:

FIG. 5 is a flow chart showing an end region confirming process in VRdisplay:

FIGS. 6A to 6C represent display examples when performing a clippingprocess from a VR image to a 180-degree image:

FIGS. 7A to 7E represent display examples when changing a clippingdirection by a right scroll;

FIGS. 8A to 8E represent display examples when changing a clippingdirection by a left scroll;

FIGS. 9A and 9B represent display examples of an end region confirmationmethod in VR display; and

FIGS. 10A to 10D represent examples of various ways to display a VRimage.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, a preferred embodiment of the present invention will bedescribed with reference to the drawings. FIG. 1 is a block diagramshowing a configuration example of a display control apparatus 100 as anexample of the electronic device. A CPU 101, a memory 102, a nonvolatilememory 103, an image processing unit 104, a display 105, an operatingunit 106, a recording medium I/F 107, an external I/F 109, and acommunication I/F 110 are connected to an internal bus 150. In addition,an audio output unit 112 and an attitude detecting unit 113 are alsoconnected to the internal bus 150. The respective units connected to theinternal bus 150 are configured so as to be capable of exchanging datawith one another via the internal bus 150.

The CPU 101 is a control unit which controls the entire display controlapparatus 100 and is constituted by at least one processor or onecircuit. The memory 102 is constituted by, for example, a RAM (such as avolatile memory using a semiconductor element). The CPU 101 controls therespective units of the display control apparatus 100 by, for example,using the memory 102 as a work memory in accordance with a programstored in the nonvolatile memory 103. The nonvolatile memory 103 storesimage data and audio data, other data, various programs that enable theCPU 101 to operate, and the like. The nonvolatile memory 103 isconstituted by, for example, a flash memory or a ROM. The CPU 101executes processes of respective units included in the display controlapparatus 100, namely, a display controlling unit, a control unit, agenerating unit, and a boundary display unit.

Under control by the CPU 101, the image processing unit 104 performsvarious types of image processing on images stored in the nonvolatilememory 103 or a recording medium 108, video signals acquired via theexternal I/F 109, images acquired via the communication I/F 110, and thelike. Image processing performed by the image processing unit 104includes an A/D conversion process and a D/A conversion process as wellas an encoding process, a compression process, a decoding process, anenlargement/reduction process (resizing), a noise reduction process, acolor conversion process, and the like of image data. The imageprocessing unit 104 also performs various types of image processing suchas panoramic development, a mapping process, and conversion ofomnidirectional images or VR images being wide-range images having videoof a wide range although not omnidirectional. The image processing unit104 may be constituted by a dedicated circuit block for performingspecific image processing. In addition, depending on a type of imageprocessing, the CPU 101 may perform the image processing in accordancewith a program without using the image processing unit 104.

The display 105 displays images, a GUI (Graphical User Interface) screenconstituting a GUI, and the like under control by the CPU 101. The CPU101 controls the respective units of the display control apparatus 100so as to generate a display control signal in accordance with theprogram and to generate a video signal to be displayed on the display105 and output the video signal to the display 105. The display 105displays a video on the basis of a generated and output video signal.Alternatively, components of the display control apparatus itself may belimited to up to an interface for outputting a video signal to bedisplayed by the display 105, and the display 105 may be constituted byan external monitor (such as a television or an HMD).

The operating unit 106 is an input device for accepting a user operationof which examples include a character information input device such as akeyboard, a pointing device such as a mouse or a touch panel, a button,a dial, a joystick, a touch sensor, and a touch pad. In this case, atouch panel 106 a is an input device which is planarly configured so asto overlap with the display 105 and to output coordinate information inaccordance with a touched position.

The recording medium 108 that is a memory card, a CD, a DVD, or the likeis mountable to and dismountable from the recording medium I/F 107.Under control by the CPU 101, the recording medium IF 107 reads datafrom and writes data to the mounted recording medium 108. The externalI/F 109 is an interface to be connected to an external device using awired cable or in a wireless manner to perform input and output of videosignals and audio signals. The communication I/F 110 is an interface forcommunicating with an external device, the Internet 111, and the like totransmit and receive various types of data such as files and commands.

The audio output unit 112 outputs audio of moving images and music data,keyboard clicks, ring tones, various notification sounds, and the like.While it is assumed that the audio output unit 112 includes an audiooutput terminal 112 a (to be described later) to which an earphone orthe like is to be connected and a speaker 112 b (to be described later),alternatively, the audio output unit 112 may output audio data to anexternal speaker by radio communication or the like.

The attitude detecting unit 113 detects an attitude (a tilt) of thedisplay control apparatus 100 relative to a direction of gravitationalforce or an attitude of the display control apparatus 100 relative torespective axes in a yaw direction, a pitch direction, and a rolldirection. On the basis of an attitude detected by the attitudedetecting unit 113, a determination can be made as to whether thedisplay control apparatus 100 is being held horizontally, heldvertically, pointed upward, pointed downward, or assuming an obliqueattitude. In addition, a determination can be made as to a presence orabsence or a magnitude of a tilt of the display control apparatus 100 ina rotation direction such as the yaw direction, the pitch direction, orthe roll direction and whether or not the display control apparatus 100has rotated in the rotation direction. One of an acceleration sensor, agyro sensor, a geomagnetic sensor, an orientation sensor, an altitudesensor, and the like or a combination of a plurality of these sensorscan be used as the attitude detecting unit 113.

The operating unit 106 includes the touch panel 106 a. The CPU 101 iscapable of detecting the following operations with respect to the touchpanel 106 a or the following states of the touch panel 106 a.

-   -   A state where a finger or a stylus previously not in touch with        the touch panel 106 a newly touches the touch panel 106 a or, in        other words, a start of a touch (hereinafter referred to as a        touch-down)    -   A state where the touch panel 106 a is being touched by a finger        or a stylus (hereinafter referred to as a touch-on)    -   A state where a finger or a stylus is moving while in touch with        the touch panel 106 a (hereinafter referred to as a touch-move)    -   A state where a finger or a stylus previously in touch with the        touch panel 106 a separates from the touch panel 106 a or, in        other words, an end of a touch (hereinafter referred to as a        touch-up)    -   A state where nothing is touching the touch panel 106 a        (hereinafter referred to as a touch-off)

When a touch-down is detected, a touch-on is simultaneously detected.Normally, after a touch-down, a touch-on is continuously detected unlessa touch-up is detected. When a touch-move is detected, a touch-on issimilarly simultaneously detected. Even when a touch-on is detected, atouch-move is not detected unless a touch position moves. A touch-off isdetected upon detection of a touch-up of all of the fingers or a styluspreviously in touch.

The CPU 101 is notified of the operations or the states described aboveas well as position coordinates where a finger or a stylus is touchingthe touch panel 106 a through an internal bus and, on the basis of thenotified information, the CPU 101 determines what kind of operation(touch operation) has been performed on the touch panel 106 a. Withrespect to a touch-move, a movement direction of a finger or a stylusmoving on the touch panel 106 a can also be determined for each of avertical component and a horizontal component on the touch panel 106 aon the basis of a change in the position coordinate. It is assumed thata determination that a slide operation has been performed is made when atouch-move of a predetermined distance or more is detected.

An operation involving quickly moving a finger on the touch panel 106 aover a certain distance while keeping the finger in touch with the touchpanel 106 a and then releasing the finger is referred to as a flick. Inother words, a flick is an operation in which a finger quickly tracesthe touch panel 106 a as though flicking on the touch panel 106 a. Adetermination that a flick has been performed can be made (adetermination that a flick has occurred following a slide operation canbe made) when a detection of a touch-move of a predetermined distance ormore at a predetermined speed or more is followed by a detection of atouch-up.

Furthermore, a touch operation involving touching a plurality oflocations (for example, two points) at the same time and bringing therespective touch positions close to each other is referred to as apinch-in while a touch operation in which the respective touch positionsare distanced from each other is referred to as a pinch-out. A pinch-outand a pinch-in are collectively referred to as a pinch operation (or,simply, a pinch). As the touch panel 106 a, a touch panel adopting anyof various systems including a resistive film system, a capacitancesystem, a surface acoustic wave system, an infrared system, anelectromagnetic induction system, an image recognition system, and anoptical sensor system may be used. Any of a system in which a touch isdetected when contact is made with the touch panel and a system in whicha touch is detected when a finger or a stylus approaches the touch panelmay be adopted.

FIG. 2 is an external view of the display control apparatus 100 which isan example of the electronic device. The display control apparatus 100is a display apparatus such as a smartphone. A display 105 is a displayunit which displays images and various types of information. The display105 is integrally constructed with the touch panel 106 a and isconfigured so as to be capable of detecting a touch operation on adisplay surface of the display 105. The display control apparatus 100 iscapable of performing VR display of a VR image (a VR content) on thedisplay 105. For example, a VR image picked up by a digital camera (anomnidirectional camera; a fully celestial camera) can be acquired viathe communication I/F 110 (a VR image transferred by wirelesscommunication can be received) and recorded in the recording medium 108.VR display (display in a VR view) of VR images recorded in the recordingmedium 108 in this manner can be performed on the display 105.Alternatively, VR display (display in a VR view) can be performed on thedisplay 105 by directly connecting a recording medium on which a VRimage picked up by a digital camera is recorded to the recording mediumI/F 107, reading the VR image from the recording medium, and playingback the read VR image. Alternatively, VR display can be performed byacquiring, via an SNS or the like, a VR image (a shared VR image)recorded on a server apparatus or the like connected to a network. Inthe present embodiment, the operating unit 106 includes the touch panel106 a and operating units 106 b, 106 c, 106 d, and 106 e. The operatingunit 106 b is a power supply button for accepting an operation to switchbetween turning a power supply of the display control apparatus 100 onand off. The operating unit 106 c and the operating unit 106 d arevolume buttons for increasing and reducing a volume of audio output fromthe audio output unit 112. The operating unit 106 e is a home button forcausing a home screen to be displayed on the display 105. The audiooutput terminal 112 a is an earphone jack that is a terminal foroutputting an audio signal to an earphone, an external speaker, or thelike. The speaker 112 b is a speaker built into a main body foroutputting audio.

The display control apparatus 100 is capable of performing VR display ofa VR image (a VR content) on the display 105. A VR image is assumed tobe an image of which VR display (display in a “VR view” display mode)can be performed. VR images are assumed to include an omnidirectionalimage (a fully celestial image) picked up by an omnidirectional camera(a fully celestial camera) and a panoramic image having a wider videorange (a wider valid video range) than a display range that can bedisplayed at one time on the display unit. VR images include movingimages and live view images (images acquired approximately in real timefrom a camera) in addition to still images. A VR image has a maximumvideo range (a maximum valid video range) corresponding to a visualfield of 360 degrees in a vertical direction (a vertical angle, an anglefrom a zenith, an elevation, a depression angle, an elevation angle, ora pitch angle) and 360 degrees in a horizontal direction (a horizontalangle, an azimuth, or a yaw angle). In addition, it is assumed that VRimages include images with a wider angle of view (a wider visual fieldrange) than an angle of view that can be photographed by an ordinarycamera or images with a wider video range (a wider valid video range)than a display range that can be displayed at one time on the displayunit even when the visual field of the images is less than 360 degreesin the vertical direction and less than 360 degrees in the horizontaldirection. For example, an image photographed by a fully celestialcamera capable of photographing a subject corresponding to a visualfield (an angle of view) of 360 degrees in the horizontal direction (ahorizontal angle or an azimuth) and 210 degrees in the verticaldirection centered on a zenith is a type of a VR image. In addition, forexample, an image photographed by a camera capable of photographing asubject corresponding to a visual field (an angle of view) of 180degrees in the horizontal direction (a horizontal angle or an azimuth)and 180 degrees in the vertical direction centered on the horizontaldirection is a type of a VR image. In other words, an image having avideo range corresponding to a field of view of 160 degrees (±80degrees) or more in both the vertical direction and the horizontaldirection and having a video range that is wider than a range that canbe visually recognized at one time by a human being is a type of a VRimage. By performing VR display (display in the “VR view” display mode)of the VR image, changing an attitude of a display apparatus (a displayapparatus for displaying the VR image) in a horizontal rotationdirection enables an omnidirectional video without any seams in thehorizontal direction (the horizontal rotation direction) to be viewed.In the vertical direction (a vertical rotation direction), although anomnidirectional video without any seams can be viewed in a range of ±105degrees with respect to directly above (the zenith), a range exceeding105 degrees from directly above becomes a blank region in which a videois not present. A VR image can be described as “an image of which avideo range is at least a part of a virtual space (a VR space)”.

VR display (a VR view) refers to a display method (a display mode) ofdisplaying a video of a visual field range in accordance with anattitude of the display apparatus among a VR image and in which adisplay range can be changed. When wearing and viewing a head mounteddisplay (HMD) that is a display apparatus, a video of a visual fieldrange in accordance with an orientation of the head of the user is to bedisplayed. For example, let us assume that a video with a viewing angle(an angle of view) centered on 0 degrees in the horizontal direction (aspecific orientation such as north) and 90 degrees in the verticaldirection (90 degrees from the zenith or, in other words, horizontal) ata certain point in time among a VR image is being displayed. When frontand back of the attitude of the display apparatus is reversed from thisstate (for example, when a display surface is changed from facing southto facing north), the display range is changed to a video with a viewingangle centered on 180 degrees in the horizontal direction (an oppositeorientation such as south) and 90 degrees in the vertical direction(horizontal) among the same VR image. This means that, in a case wherethe user is viewing the HMD, when the user turns his or her head fromnorth to south (in other words, when the user turns around), the videodisplayed on the HMD also changes from a northward video to a southwardvideo. Such a VR display enables the user to be provided with asensation (a sense of immersion) as though the user is visually presentinside the VR image (inside the VR space). A smartphone mounted to VRgoggles (a head mount adapter) can be considered a type of HMD.

It should be noted that a display method of a VR image is not limited tothe method described above. A configuration may be adopted in which adisplay range is moved (scrolled) in accordance with a user operationwith respect to a touch panel, a directional button, or the like insteadof an attitude change. A configuration may be adopted in which, evenwhen a VR image is displayed by VR display (displayed in the “VR view”display mode), a display range can be changed in accordance with atouch-move with respect to the touch panel, a drag operation withrespect to a mouse or the like, a depression of a directional button, orthe like in addition to changing the display range in accordance with anattitude change.

In the present embodiment, an operation and a process of clipping a VRimage with a video range (a valid video range) corresponding to a fieldof view of 180 degrees in both the vertical direction and the horizontaldirection from a VR image with a video range (a valid video range)corresponding to a maximum field of view of 360 degrees in both thevertical direction and the horizontal direction will be described. Itshould be noted that, in the following description, a VR image with avideo range (a valid video range) corresponding to a maximum field ofview of 360 degrees in both the vertical direction and the horizontaldirection will be referred to as a 360-degree VR image. In addition, aVR image with a video range (a valid video range) corresponding to afield of view of 180 degrees in both the vertical direction and thehorizontal direction will be referred to as a 180-degree VR image.Conceptually, the process of clipping a 180-degree VR image from a360-degree VR image is a process of clipping a hemisphere from a virtualsphere onto which a video corresponding to 360 degrees has been entirelymapped. Clipping as described in the present embodiment is not a processof clipping an ordinary rectangular image (a planar image) but, rather,a process of clipping an image that can be viewed as a VR image (animage that can be mapped onto a sphere and viewed in a VR view) evenafter the clipping. Since the image after the clipping has a valid videorange corresponding to a field of view of 180 degrees in both thevertical direction and the horizontal direction, when the image afterthe clipping is mapped onto a sphere, a valid video (such as aphotographed image) corresponding to a hemisphere is to be mapped. Theremaining hemisphere becomes an invalid video range to be filled inmonochrome or by a predetermined pattern or onto which some kind ofcomplemented video is mapped. When viewing such a clipped VR image in aVR view, the user can visually recognize the image in a forward rangewith a width of 180 degrees (a range of 90 degrees in both the verticaldirection and the horizontal direction from center). Reasons forperforming such a clipping process include the following.

First, a data amount of an image can be reduced. A clipped 180-degree VRimage has a smaller data amount than a 360-degree VR image. Therefore,capacity of a recording medium for saving VR images can be preventedfrom becoming oppressed. In addition, an amount of communication dataduring transmission and reception and processing load during display canbe reduced and, proportionally, effects including an improvement inprocessing speed or response speed and a reduction in power consumptioncan be produced.

Second, unnecessary subjects can be deleted. For example, in most caseswhere a VR image having a valid video range corresponding to a field ofview of 360 degrees is picked up, it is unavoidable that thephotographer himself/herself is included in the VR image. However, in acase where an object that the photographer wishes to photograph is alandscape or the like and not the photographer himself/herself, thephotographer himself/herself is an unnecessary subject and constitutesan unintentional presence. The inclusion of an unintentional videoprevents a viewer from focusing his or her attention on a video such asa landscape which the photographer considers to be a theme. In addition,information to be concealed such as the face of a passerby and thelicense plate of a passing vehicle is often included and, in many cases,the inclusion of such information is undesirable from the perspectivesof privacy and security. By performing a clipping process to clip arange intended by the photographer and discard video of otherunnecessary ranges, such problems can be avoided.

Third, a physical burden on a user viewing a VR image in a VR view canbe reduced. When viewing a 360-degree VR image in a VR view, video ispresent even behind the user viewing the VR image. While videocorresponding to forward 180 degrees can be roughly panoramically viewedby a viewer by simply turning his or her head, in order to look behind,the view must twist his or her torso or change his or her standingdirection, thereby placing a relatively high physical burden on theviewer. Such a body movement to look behind is unsuitable for viewingwhen seated unless the viewer is in a swivel chair. In comparison, sincea 180-degree VR image can be roughly panoramically viewed by simplyturning one's head, the physical burden placed on the viewer whenviewing the VR image is relatively small. If the viewer is made aware ofthe fact that a VR image is a 180-degree VR image by a guide display ora display indicating that a range exceeding 180 degrees is an invalidregion, the viewer does not even attempt to look behind. Therefore, theviewer can be prevented from being forced to perform a body movementsuch as looking behind.

Fourth, a 360-degree VR image can be displayed in accordance with arecording format for a 180-degree VR image or the like. When a recordingformat of a 360-degree VR image differs from the recording format for a180-degree VR image, the 360-degree VR image cannot be viewed by areproducing apparatus (reproduction application software) which onlysupports viewing 180-degree VR images. However, generating a 180-degreeVR image by clipping the 180-degree VR image from a 360-degree VR imageenables the VR image to be viewed by a reproducing apparatus which onlysupports viewing 180-degree VR images. In addition, in a case where bothviewing 180-degree VR images and viewing 360-degree VR images aresupported, switching operations for switching between a mode for viewing180-degree VR images and a mode for viewing 360-degree VR images must beperformed, switching operations may be a hassle. However, generating a180-degree VR image by clipping the 180-degree VR image from a360-degree VR image enables a plurality of VR images to be switched andviewed in succession without having to switch to the display mode for180-degree VR images. Furthermore, a display mode (a binocular VR view)is envisaged which displays two 180-degree VR images including aright-eye 180-degree VR image and a left-eye 180-degree VR imagephotographed by two camera units installed facing a subject in a samedirection and separated by a distance corresponding to parallax in sucha manner that the two 180-degree VR images can be stereoscopicallyviewed. By clipping a right-eye 180-degree VR image and a left-eye180-degree VR image from a 360-degree VR image and recording the180-degree VR images in advance, the 180-degree VR images can also beviewed in such a binocular VR view.

While an example in which a 180-degree VR image is generated by clippingthe 180-degree VR image from a 360-degree VR image will be described inthe present embodiment, the present embodiment is also applicable toprocesses for VR images with other viewing angles as long the processesare for clipping a VR image with a valid video range that is narrowerthan a valid video range of the VR image prior to the clipping. In otherwords, the VR image prior to clipping is not limited to a 360-degree VRimage and the VR image after the clipping is not limited to a 180-degreeVR image.

In each process described below, an example of advancing the process bydisplaying various touch buttons and accepting a touch operation withrespect to the various touch buttons as a user operation will bedescribed. Specifically, a touch operation of accepting an instructionwith respect to various touch buttons may be a touch-up from a touchresponse region corresponding to a touch button or a touch-down withrespect to a touch response region corresponding to a touch button. Inaddition, each instruction described as an instruction to be accepted byan operation to a touch button to be described later is not limitedthereto and may be accepted by other user operations. For example, auser operation for accepting an instruction may be an operation withrespect to each physical button, an operation of selecting a displayitem using a direction key and depressing an enter key, an operation ofselecting a display item using a mouse, a voice command, or the like.

Hereinafter, operations of a practical example of the present inventionwill be described with reference to FIGS. 3 to 8A to 8E.

FIG. 3 is a flow chart showing an example of a clipping process from a360-degree VR image to a 180-degree image by the display controlapparatus 100. The processes of this flow chart are realized as the CPU101 deploys a program stored in the nonvolatile memory 103 on the memory102 and executes the program.

The processes shown in FIG. 3 may be started when a power supply of thedisplay control apparatus 100 is turned on and a VR image (a VR content)is selected from images recorded on the recording medium 108 and imagesacquired from a communication destination.

In S301, the CPU 101 loads a VR image to be a display object from therecording medium 108 or a communication destination via thecommunication I/F 110. The CPU 101 acquires information indicating avideo range (a valid video range) attached as attribute information ofthe VR image, and displays the VR image by flat display in which thevalid video range is fitted into a rectangle. The valid video range ofthe VR image subjected to flat display on the display 105 in S301 is anexample of the first valid video range. While a VR image is displayed bymapping the VR image onto a virtual sphere in a VR view, in flatdisplay, the VR image is displayed by fitting the entire VR image into arectangle by equidistant cylindrical projection or the like. Therefore,in portions near ends of the rectangle and the like, a subject is moredistorted than when the subject is displayed in a VR view.

The information indicating a valid video range refers to informationdescribing angular ranges in the vertical direction and in thehorizontal direction of a valid video included in the VR image. Theinformation indicating a valid video range may be information thatenables a valid video range of the VR image to be specified and may beangle information represented by an angle of view, a viewing angle, anazimuth, an elevation, a depression angle, an elevation angle, asteradian, or the like or positional information such as the numbers ofvertical and horizontal pixels or coordinates.

Alternatively, the information indicating a valid video range may bemodel information of the camera used to capture the VR image (aphotographable range can be specified by specifying a model),information on zoom during photography, and the like. When the CPU 101acquires the information indicating a valid video range, the CPU 101also identifies (calculates) an invalid video range (a non-video range)based on differences from vertical and horizontal 360 degrees.Conversely, information indicating an invalid video range may beacquired from the attribute information of the VR image and a validvideo range may be calculated from the information indicating theinvalid video range, or both information indicating a valid video rangeand information indicating an invalid video range may be acquired fromthe attribute information of the VR image.

FIG. 6A shows a display example (a display example on a first screen) offlat display in S301. Since the image uses equidistant cylindricalprojection, unlike the actual subject, the image is distorted. The usercan make a transition to a clipping mode of a VR image 601 by a buttonoperation involving selecting a clipping button 602. The clipping button602 is an operation button for accepting an instruction for setting aclipping range from the user. In addition, an instruction icon formaking a transition to a VR view may be displayed. In accordance with anoperation with respect to the instruction icon, a transition can be madefrom flat display to a VR view with respect to a same image. It shouldbe noted that, when a touch-move is performed on a display region of theVR image 601 during the flat display in S301, it is assumed that the VRimage 601 is switched to another image (switched to display of a nextimage or a previous image in a predetermined order such as an order offile names). The flat display in S301 is a screen used by the user toidentify an image to be a display object or an editing object. Whenimages are to be switched by a touch-move, while the VR image 601 isslid (scrolled) in a direction of the touch-move so as to follow thetouch-move, a non-image region colored in black or the like or an imageafter the switch is displayed on an outer side of ends of the VR image601 shown in FIG. 6A. In other words, unlike a scroll in a clippingdirection setting process shown in FIG. 4, a cyclical scroll thatprevents ends from being identified is not performed.

In S302, the CPU 101 determines whether or not a touch operation withrespect to the clipping button 602 has been performed or, in otherwords, whether or not an instruction to set a clipping range withrespect to the touch panel 106 a has been issued. When an instruction toset a clipping range has been issued, the CPU 101 advances to S303, butin a case where an instruction to set a clipping range has not beenissued, the CPU 101 ends the process. When the CPU 101 advances to S303,a transition is made to a mode of setting a clipping range for clippingthe VR image at 180 degrees.

In S303, the CPU 101 displays a selection screen for selecting whetheror not to perform zenith correction, receives a parameter related tozenith correction having been input to the touch panel 106 a, andperforms zenith correction when it is desired to do so. FIG. 6B shows adisplay example of the selection screen for selecting whether or not toperform zenith correction which is displayed in S303. A zenithcorrection selection dialog 603 shown in FIG. 6B is displayed byperforming a button operation of selecting the clipping button 602. Thezenith correction selection dialog 603 includes a button 604 and abutton 605. In accordance with a button operation by the user ofselecting the button 604, correction of a zenith axis is executed beforethe clipping process of the VR image. Alternatively, when the button 605is selected, correction of the zenith axis is not executed before theclipping process of the VR image.

In S304, the CPU 101 displays a UI for clipping on the display 105. Inother words, in accordance with the instruction to set a clipping rangefrom the user, the display screen described in S301 is switched to a UIscreen for clipping (switching control). FIG. 6C shows a display exampleof the UI for clipping (a display example of a second screen) which isdisplayed in S304. A clipping region frame 606 is an indicator thatindicates a 180-degree region to be clipped from the VR image. Theclipping region frame 606 is displayed on the VR image (on the VRcontent) being displayed by flat display in a rectangular region. Itshould be noted that the clipping region frame 606 does not particularlyindicate an accurate clipping range and merely indicates a rough range.A non-clipping region 608 indicates a region not included in the180-degree region to be clipped.

A button operation of selecting a clipping determination (completion)button 611 by the user causes a clipping region 607 enclosed by theclipping region frame 606 to be set as the clipping region. A buttonoperation of selecting a zenith clipping button 612 causes a 180-degreeregion centered on a zenith direction (an upward direction of the VRcontent) to be set as the clipping region. A button operation ofselecting a nadir clipping button 613 causes a 180-degree regioncentered on a nadir direction (a downward direction of the VR content)to be set as the clipping region. A button operation of a return button609 causes a screen of flat display in FIG. 6A to be displayed. On thebasis of the set clipping region, the CPU 101 can generate an edited VRcontent having a valid video range that is narrower than the valid videorange prior to clipping. In this manner, the CPU 101 does not scroll aVR content in the vertical direction, and clipping of the VR content inthe vertical direction is performed by a button operation (an operationnot involving a scrolling operation) by the user.

In S305, the CPU 101 determines whether or not a touch-down has beendetected with respect to a central region on an inner side of theclipping region 607 among a display region of the VR image beingdisplayed by flat display on the display 105. The central region on theinner side of the clipping region 607 is a region on an inner side by apredetermined width or more of left and right portion of the clippingregion frame 606 which represent left and right boundaries of theclipping region 607. When a touch-down with respect to the centralregion is detected, the CPU 101 advances to S306, but in a case where atouch-down with respect to the central region is not detected, the CPU101 advances to S307.

In S306, the CPU 101 performs the clipping direction setting process. Itshould be noted that the clipping direction (a range of the image to beclipped) is set on the basis of the region enclosed by the clippingregion frame 606 shown in FIG. 6C. The valid video range of the regionto be clipped is an example of the second valid video range. The CPU 101performs control such that at least a center of the region enclosed bythe clipping region frame 606 shown in FIG. 6C and a center of theactual valid video range to be clipped (the second valid video range)coincide or approximately coincide with each other. However, since theclipping region frame 606 shown superimposed on the flat display doesnot accurately express a clipping boundary of a hemisphere, the rangeindicated by the clipping region frame 606 and the actual valid videorange to be clipped (the second valid video range) do not completelycoincide with each other. For example, the range of the region enclosedby the clipping region frame 606 may be made larger than the actualvalid video range (180 degrees) to be clipped. The clipping directionsetting process will be described later with reference to FIG. 4.

In S307, the CPU 101 performs a VR end region confirming process. The VRend region confirming process will be described later with reference toFIG. 5.

In S308, the CPU 101 determines whether or not a touch operation withrespect to the clipping determination button 611 has been performed or,in other words, whether or not a clipping instruction has been issued.When a clipping instruction has been issued, the CPU 101 advances toS309, but in a case where a clipping instruction has not been issued,the CPU 101 ends the process.

In S309, the CPU 101 clips an image in a range of 180 degrees in theclipping direction set in S306 and saves the clipped image in a VR180-degree format in the recording medium 108. In other words, the CPU101 records the clipped VR image in the recording medium 108.

The clipped VR image (an image file that can be displayed in a VR view)to be recorded in S309 will be described. When the clipped VR image is astill image, a single file is generated and recorded, the single filestoring, in a multi-picture format, a plurality of images created bydeforming an image within a clipping range by equidistant cylindricalprojection and fitting the deformed image into a rectangle. When a360-degree VR image prior to clipping (an original image) is displayedas an ordinary planar image instead of a VR view, an image drawn byequidistant cylindrical projection in a rectangle such as that shown inFIG. 10A is obtained. The image is displayed in this manner in S301.When a 180-degree range is clipped from the original image by theprocess described with reference to FIG. 4, an image drawn byequidistant cylindrical projection in an approximate square such as thatshown in FIG. 10B is recorded when the 180-degree range is displayed asan ordinary planar image instead of a VR view. In doing so, two of asame image obtained by copying the image inside the clipping range arerespectively recorded in a same file as a right-eye image and a left-eyeimage even when there is no parallax. Alternatively, two 180-degreeranges may be clipped and generated from one 360-degree VR image priorto clipping so as to have a pseudo-parallax (in other words, so thatranges of the left-eye image and the right-eye image slightly differfrom each other). For example, a file name of a VR image of a clippedstill image is “123456.vr.jpg”, and a character string made up of threecharacters “.vr“is described before the extension”.jpg”. When an imagesaved as a still image file in this manner is reproduced and displayedin a VR view, the image shown in FIG. 10B is mapped onto a hemisphereand displayed. FIG. 10D shows an example in which a VR image file of aclipped still image is reproduced and displayed in a binocular VR view(a display example on the display 105 in a state where the display 105is not mounted on VR goggles).

When the clipped VR image is a moving image, a moving image file isgenerated and recorded, the moving image file having videos (movingimages) obtained by mapping the image within the clipping range to theinside of a sphere or an ellipse instead of equidistant cylindricalprojection arranged horizontally (side by side) inside the video of asingle moving image. In doing so, two of a same image obtained bycopying the image within the clipping range are respectively arrangedhorizontally and recorded as a right-eye image and a left-eye image evenwhen there is no parallax. When such a moving image is displayed as anordinary planar image instead of a VR view, the moving image isdisplayed as shown in FIG. 10C. Alternatively, two 180-degree ranges maybe clipped and generated from one 360-degree VR image prior to clippingso as to have a pseudo-parallax (in other words, so that ranges of theleft-eye image and the right-eye image slightly differ from each other).For example, a file name of a VR image of a clipped moving image is“123456.vr.mp4”, and a character string made up of three characters“.vr“is described before the extension”.mp4”. A display example of agiven frame of a VR image file of a clipped moving image when reproducedand displayed in a binocular VR view (a display example on the display105 in a state where the display 105 is not mounted on VR goggles) issimilar to the example shown in FIG. 10D.

FIG. 4 is a flow chart showing an example of the clipping directionsetting process of S306 shown in FIG. 3. The clipping direction settingprocess is realized as the CPU 101 deploys a program stored in thenonvolatile memory 103 on the memory 102 and executes the program.

In S401, the CPU 101 determines whether or not a right scroll has beeninput with respect to the touch panel 106 a. The input of a right scrollis an operation of scrolling an image rightward and corresponds to ascroll instruction in a rightward direction. The input of a right scrollis, for example, a touch-move that moves to the right or an operation ofsliding a scroll bar (not illustrated) toward the left. When a rightscroll is input, the CPU 101 advances to S402, but in a case where aright scroll is not input, the CPU 101 advances to S404.

In S402, when the CPU 101 receives an input of a right scroll from theuser, the CPU 101 scrolls (slides) the image toward the right. The CPU101 sequentially displays an image region of the display 105corresponding to a rightward scroll amount in accordance with the scrollfrom a left end of the display 105 toward the right. It should be notedthat a position of the clipping region frame 606 on the screen is fixedand the position is not scrolled even when a scroll is input.

In S403, the CPU 101 displays a region on a leftward direction sideexcluding an image region on a right side corresponding to the scrollamount by moving the region rightward by the scroll amount. In otherwords, the CPU 101 scrolls the VR content displayed by flat display inthe rectangular region of the display 105 rightward. In addition, inaccordance with the scroll, the CPU 101 sequentially displays an imageregion corresponding to the rightward scroll amount in the VR contentfrom a leftward end in an opposite direction to the rightward directiontoward the right in the rectangular region.

FIGS. 7A to 7E show a display example in accordance with the scrollsperformed in S402 and S403. A right scroll 701 shown in FIG. 7Brepresents a scroll input in a rightward direction with respect to FIG.7A. When the right scroll 701 is input, the processes of S402 and S403shown in FIG. 4 are executed. For example, let us assume that, in S401,the user has scrolled toward the right end side of the display 105 by ascroll amount D1. In S402, the CPU 101 sequentially displays an imageregion 7011 which is enclosed by a dotted line and which corresponds tothe scroll amount (D1) from the left end of the display 105 toward theright. Next, in S403, the CPU 101 displays a left-side region 7012enclosed by a dashed-dotted line and excluding the image region 7011corresponding to the scroll amount (D1) by moving the region 7012rightward by the scroll amount (D1). Once the processes are executed,the display 105 changes to the state shown in FIG. 7C. A buttonoperation of selecting a clipping determination (completion) button 611by the user causes a clipping region 702 enclosed by the clipping regionframe 606 to be set as the clipping region.

A right scroll 703 shown in FIG. 7D represents a scroll input in arightward direction with respect to FIG. 7C. When the right scroll 703is input, the processes of S402 and S403 shown in FIG. 4 are executed ina similar manner to when the right scroll 701 was input, and the display105 changes to the state shown in FIG. 7E. A button operation ofselecting a clipping determination (completion) button 611 by the usercauses a clipping region 704 enclosed by the clipping region frame 606to be set as the clipping region.

As described above, when the scrolls performed in S402 and S403 involvescrolling an image to the right, a portion which is scrolled all the wayto outside of a right-side display region and which disappears isdisplayed so as to successively enter the display region from a left endthereof toward the right. In addition, an endless scroll can be realizedwithout reaching an end by repetitively inputting right scrolls. Inother words, a cyclical scroll display of a same content which does notmake the user aware of boundaries such as left and right ends of the VRimage 601 can be realized. Accordingly, a portion positioned at an endof the display region in the display example shown in FIG. 6A can now bepositioned at the center of the clipping region frame 606. In addition,an instruction to make a seamless 180-degree region centered on theposition (a range with a circumference of 90 degrees which is centeredon the position) the VR image after clipping can be issued. It should benoted that, since cyclical scroll display enables an arbitraryorientation of the VR image to be designated as center, the scrolldisplay need not be endlessly cyclical and may enable at least one cycleto be made.

In S404, the CPU 101 determines whether or not a left scroll toward anopposite side to the rightward direction has been input with respect tothe touch panel 106 a. The input of a left scroll is an operation ofscrolling an image to the left and corresponds to a scroll instructionin a leftward direction. The input of a left scroll is, for example, atouch-move that moves to the left or an operation of sliding a scrollbar (not illustrated) toward the right. When a left scroll is input, theCPU 101 advances to S405, but in a case where a left scroll is notinput, the CPU 101 advances to S407.

In S405, when the CPU 101 receives an input of a left scroll from theuser, the CPU 101 scrolls (slides) the image toward the left. The CPU101 sequentially displays an image region of the display 105corresponding to a leftward scroll amount in accordance with the scrollfrom a right end of the display 105 toward the left. It should be notedthat a position of the clipping region frame 606 on the screen is fixedand the position is not scrolled even when a scroll is input.

In S406, the CPU 101 displays a region on a rightward direction sideexcluding an image region on a left side corresponding to the scrollamount by moving the region leftward by the scroll amount. In otherwords, the CPU 101 scrolls the VR content displayed by flat display inthe rectangular region of the display 105 leftward. In addition, inaccordance with the scroll, the CPU 101 sequentially displays an imageregion corresponding to the leftward scroll amount in the VR contentfrom a rightward end in an opposite direction to the leftward directiontoward the left in the rectangular region.

FIGS. 8A to 8E show a display example in accordance with the scrollsperformed in S402 and S403. A left scroll 801 shown in FIG. 8Brepresents a scroll input in a leftward direction with respect to FIG.8A. When the left scroll 801 is input, the processes of S405 and S406shown in FIG. 4 are executed. For example, let us assume that, in S404,the user has scrolled contrary to FIG. 7B toward the left end side ofthe display 105 by a scroll amount D2. In S405, the CPU 101 sequentiallydisplays an image region 8011 which is enclosed by a dotted line andwhich corresponds to the scroll amount (D2) from the right end of thedisplay 105 toward the left. Next, in S406, the CPU 101 displays aleft-side region 8012 enclosed by a dashed-dotted line and excluding theimage region 8011 corresponding to the scroll amount (D2) by moving theregion 8012 leftward by the scroll amount (D2). Once the processes areexecuted, the display 105 changes to the state shown in FIG. 8C. Abutton operation of selecting a clipping determination (completion)button 611 by the user causes a clipping region 802 enclosed by theclipping region frame 606 to be set as the clipping region.

A left scroll 803 shown in FIG. 8D represents a scroll input in aleftward direction with respect to FIG. 8C. When the left scroll 803 isinput, the processes of S405 and S406 shown in FIG. 4 are executed in asimilar manner to when the left scroll 801 was input, and the display105 changes to the state shown in FIG. 8E. A button operation ofselecting a clipping determination (completion) button 611 by the usercauses a clipping region 804 enclosed by the clipping region frame 606to be set as the clipping region.

As described above, when the scrolls performed in S404 and S405 involvescrolling an image to the left, a portion which is scrolled all the wayto outside of a left-side display region and which disappears isdisplayed so as to successively enter the display region from a rightend thereof toward the left. In addition, an endless scroll can berealized without reaching an end by repetitively inputting left scrolls.In other words, a cyclical scroll display of a same content which doesnot make the user aware of boundaries such as left and right ends of theVR image 601 can be realized. Accordingly, a portion positioned at anend of the display region in the display example shown in FIG. 6A cannow be positioned at the center of the clipping region frame 606. Inaddition, an instruction to make a seamless 180-degree region centeredon the position (a range with a circumference of 90 degrees which iscentered on the position) the VR image after clipping can be issued.

In S407, the CPU 101 determines whether or not a touch operation withrespect to a reset button 610 has been performed or, in other words,whether or not an operation of a reset instruction of a clippingdirection has been accepted. When a reset instruction has been issued,the CPU 101 advances to S408, but in a case where a reset instructionhas not been issued, the CPU 101 advances to S409. A reset clippingdirection position is a setting of the clipping direction set during theprocess of S304 shown in FIG. 3. In S408, the CPU 101 sets the clippingdirection to an initial position.

In S409, the CPU 101 determines whether or not a touch operation withrespect to the clipping determination button 611 has been performed or,in other words, whether or not an operation of a determinationinstruction of a clipping direction has been accepted. When adetermination instruction of a clipping direction has been issued, theCPU 101 ends the process, but in a case where a determinationinstruction of a clipping direction has not been issued, the CPU 101proceeds to S401. It is assumed that, in a case where a determination ofYes is made in S409, a determination of Yes is also made in S308. Inother words, in the present embodiment, the determination instruction ofa clipping direction and the clipping instruction are the sameinstruction. However, the determination instruction of a clippingdirection and the clipping instruction may be configured so as to beperformed by separate operations. For example, a preview displayindicating a range of the VR image after clipping may be performed inresponse to an issuance of the determination instruction of a clippingdirection and, subsequently, an image file of the clipped VR image maybe generated in accordance with an issuance of the clipping instruction.

It should be noted that an orientation of a range of the VR image afterclipping among the VR image prior to the clipping can be designated byscrolls in the horizontal direction described in S401 to S406. Anelevation/depression angle of the VR image after clipping is fixed to arange from zenith to nadir with an elevation/depression angle of 180degrees centered on the horizontal direction (a direction 90 degreesfrom zenith: after zenith correction, a direction 90 degrees from thezenith after correction). Accordingly, the hassle of designating aclipping range can be prevented. In addition, the VR image afterclipping can be set to 180 degrees centered on front (the horizontaldirection) from the point of view of the user viewing the VR image andconfigured such that a rear-side range beyond zenith or a rear-siderange beyond nadir is not included in the VR image. Accordingly, viewingin uncomfortable postures when the VR image is viewed in a VR view suchas arching backward to look back can be prevented.

When the zenith clipping button 612 is touched, the clipping range isset to a range of 90 degrees in all directions centered on zenith (afterzenith correction, a direction 90 degrees from the zenith aftercorrection) regardless of a range indicated by the clipping region frame606 (an orientation designated by the user). This represents an upperhemispherical range when a virtual sphere is equally vertically divided.In addition, when the nadir clipping button 613 is touched, the clippingrange is set to a range of 90 degrees in all directions centered onnadir (after zenith correction, a direction 90 degrees from the nadirafter correction) regardless of a range indicated by the clipping regionframe 606 (an orientation designated by the user). This represents alower hemispherical range when a virtual sphere is equally verticallydivided. When the zenith clipping button 612 or the nadir clippingbutton 613 is touched, it is assumed that a clipping instruction hasalso been issued and processes up to generation of a file of the clippedVR image in S309 shown in FIG. 3 are performed. Accordingly, unnaturalclipping which generates a hemisphere resulting from diagonally cuttingthe virtual sphere prior to the clipping or the like can be prevented.In other words, a VR image after clipping which provides a panoramicview of the entire sky above the horizon or a VR image after clippingwhich provides a panoramic view of an entire room as seen from theceiling or a panoramic view of all of the dishes on a table as seen fromabove the table can be readily generated.

FIG. 5 is a flow chart showing an example of the VR end regionconfirming process of S307 shown in FIG. 3. The VR end region confirmingprocess is realized as the CPU 101 deploys a program stored in thenonvolatile memory 103 on the memory 102 and executes the program. TheVR end region confirming process is a process of enabling the user toconfirm end regions of a VR image to be clipped by performing VR displayof boundaries of a region to be clipped and regions including suchboundaries.

In S501, the CPU 101 determines whether or not left and right endregions of the clipping region 607 instead of the center regiondescribed earlier among a display region of the VR image being displayedhas been touched as a confirmation instruction of an end of a clippingrange. The left and right end regions refer to a region (a left endregion) within a predetermined width from a left side including a leftside of the clipping region frame 606 and a region (a right end region)within a predetermined width from a right side including a right side ofthe clipping region frame 606 which are left and right boundaries of theclipping region 607. When the left and right end regions have beentouched, the CPU 101 advances to S502, but in a case where the left andright end regions have not been touched, the CPU 101 ends the process.The confirmation instruction of an end of a clipping range is notlimited thereto. For example, an icon for issuing the confirmationinstruction of an end of a clipping range may be displayed and the iconmay be touched to issue the confirmation instruction of an end of aclipping range. Alternatively, a side to the left of the left side ofthe clipping region frame 606 or a side to the right of the right sideof the clipping region frame 606 among the display region of the VRimage being displayed may be touched to issue the confirmationinstruction of an end of a clipping range.

In S502, the CPU 101 reads the VR image acquired in S301. In addition,the CPU 101 acquires information indicating a video range (a valid videorange) to be attached as attribute information of the VR image. Theinformation indicating a valid video range refers to informationdescribing angular ranges in the vertical direction and in thehorizontal direction of a valid video included in the VR image. Theinformation indicating a valid video range may be information thatenables a valid video range of the VR image to be specified and may bean angle of view, a viewing angle, an azimuth, an elevation, the numbersof horizontal pixels, or positional information such as coordinates.Alternatively, the information indicating a valid video range may bemodel information of the camera used to capture the VR image (aphotographable range can be specified by specifying a model),information on zoom during photography, and the like.

When the CPU 101 acquires the information indicating a valid videorange, the CPU 101 also identifies (calculates) an invalid video range(a non-video range) based on differences from vertical and horizontal360 degrees. Conversely, information indicating an invalid video rangemay be acquired from the attribute information of the VR image and avalid video range may be calculated from the information indicating theinvalid video range, or both information indicating a valid video rangeand information indicating an invalid video range may be acquired fromthe attribute information of the VR image.

In S503, the CPU 101 determines whether or not the right end regiondescribed earlier has been touched. When the right end region has beentouched, the CPU 101 advances to S504, but in a case where the right endregion has not been touched, the CPU 101 advances to S505.

In S504, the CPU 101 performs a display process of the right end regionof the VR image acquired in S502 on the basis of a display range withinthe range of the clipping region frame 606 among the VR image. Originaldata (an original image) of the VR image is an image which is distorteddue to the use of, for example, equidistant cylindrical projection, andwhich is an image in a format that enables a position of each pixel tobe associated with coordinates on a surface of a sphere. The originalimage of the VR image is mapped onto a sphere, and a part of the sphereis clipped and displayed. In other words, the image displayed in S504 isan image obtained by clipping and enlarging a part of the VR image andis an image in which distortion of the original image has been removed(or reduced). In this case, the CPU 101 sets a valid range of thedisplayed VR image to a range of orientations of 90 degrees to both theleft and the right of a center on a center line of a region enclosed bythe clipping region frame 606 (a total of 180 degrees) and to a range ofan elevation/depression angle from zenith to nadir. The CPU 101 setsother ranges as outside the valid range (invalid ranges, excluded rangesthat are outside of the clipping range). By displaying the right-sideend region so as to be centered on a boundary line between the validrange and the invalid range, the CPU 101 enables a state of an end onthe boundary line between the right-side valid range and the invalidrange to be confirmed.

In S505, the CPU 101 performs a display process of the left end regionof the VR image acquired in S502 on the basis of a display range withinthe range of the clipping region frame 606 among the VR image. A samedisplay method as that in S504 is used. In this case, by displaying theleft end region so as to be centered on a boundary line between thevalid range and the invalid range, the CPU 101 enables a state of an endon the boundary line between the left-side valid range and the invalidrange to be confirmed.

In S506, on the basis of information notified from the touch panel 106a, the CPU 101 determines whether or not a touch-move has been detected.When a touch-move has been detected, the CPU 101 advances to S507, butin a case where a touch-move has not been detected, the CPU 101 advancesto S508.

In S507, the CPU 101 changes the display range of the VR image inaccordance with a direction of a slide operation by the touch-move (adirection of the touch-move). For example, when it is determined thatthe direction of the touch-move is a leftward direction, by shifting(sliding) the display range of the VR image leftward by an amountcorresponding to the touch-move, the user can perform the operationwhile experiencing an operation feeling as though moving the entire VRimage. At this point, a display position on the display 105 of a regionoutside of the clipping range is fixed. Therefore, a relativerelationship between the region outside of the clipping range and thedisplayed range to be inside the clipping range is shifted by thetouch-move and a clipping range (a clipping direction) with respect tothe VR image prior to the clipping is changed. In this manner, theclipping range can be set by finely adjusting an extent of a region tobe fitted into the clipping range while confirming the boundary of theclipping range.

In S508, the CPU 101 determines whether or not an operation for aclipping instruction has been performed with respect to the touch panel106 a. When a clipping instruction has been issued, the CPU 101 advancesto S509, but in a case where a clipping instruction has not been issued,the CPU 101 advances the process to S510.

In S509, the CPU 101 clips an image in a range of 180 degrees in theclipping direction set in S508 and saves the image in a VR 180-degreeformat in the recording medium 108. The generated image is the same asthat described in S309.

In S510, on the basis of information notified from the touch panel 106 athe CPU 101 determines whether or not a return instruction has beenissued by a touch operation on a return button. When a returninstruction has been issued, the CPU 101 advances the process to S511,but in a case where a return instruction has not been issued, the CPU101 returns the process to S506.

In S511, the CPU 101 reads the VR image acquired in S502, acquiresinformation indicating a video range (a valid video range) attached asattribute information of the VR image, and displays the VR image by flatdisplay in which the valid video range is fitted into a rectangle. Whenthe display range has been changed in S507, a center position of theimage to be displayed by flat display is set on the basis of informationon the center of the display range after the change. After display, theCPU 101 ends the VR end region confirming process.

End confirmation display of the VR end region confirming process willnow be described with reference to FIGS. 9A and 9B. This display is thedisplay performed in S504 and S505 shown in FIG. 5.

FIG. 9A is a diagram showing a state where a right end side is displayedwhen VR display of a VR image is performed. FIG. 9A is displayed by, forexample, touching a position above a frame on the right side of theclipping region frame 606 or touching the right-side non-clipping region608 shown in FIG. 6C. Touching a position above the frame on the rightside of the clipping region frame 606 or touching the right-sidenon-clipping region 608 is an example of the “confirmation instruction”by the user. A confirmation instruction is an operation of designating aboundary or a region including the boundary to be a confirmation objectin a region enclosed by the clipping region frame 606. A right-sidevalid range 901 and an invalid range 902 are displayed by VR display (aVR view). VR display (a VR view) is a display mode which is mapped ontoa virtual sphere and which has been deformed from flat display.Therefore, a distorted portion in flat display can be accuratelyconfirmed in a shape where the distortion has been corrected. Theinvalid range 902 may be displayed while being masked by black, white,or a pattern display. Displaying a boundary at approximately centerenables a state of the right-side boundary line of the VR image to beconfirmed. Arranging a return button 903 and a clipping button 904 onthe invalid range 902 enables the user to confirm a region including theright-side valid range 901 and the right-side boundary line of the VRimage and to identify boundaries.

FIG. 9B is a diagram showing a state where a left end side is displayedwhen VR display of a VR image is performed. FIG. 9B is displayed by, forexample, touching a position above a frame on the left side of theclipping region frame 606 or touching the left-side non-clipping region608 (a confirmation instruction) shown in FIG. 6C. A left-side validrange 905 and an invalid range 906 are displayed by VR display. Theinvalid range 906 may be displayed while being masked by black, white,or a pattern display. Displaying a boundary at approximately centerenables a state of the left-side boundary line of the VR image to beconfirmed. In a similar manner to FIG. 9A, arranging the return button903 and the clipping button 904 on the invalid range 906 enables theuser to confirm a region including the left-side valid range 905 and theleft-side boundary line of the VR image and to identify boundaries.

While an example where a position above a frame of the clipping regionframe 606 or the non-clipping region 608 is touched as a confirmationinstruction from the user has been demonstrated, a confirmationinstruction is not limited thereto. For example, the CPU 101 may displayan icon for designating a boundary to be a confirmation object in aregion enclosed by the frames of the clipping region frame 606 on thetouch panel 106 a. By touching the icon, the user can confirm a regionincluding the boundary on a screen shown in FIG. 9A or 9B and identifythe boundary. The screen shown in FIG. 9A or 9B represents virtualinformation that is displayed for the purpose of having the user confirminformation on a boundary and identify the boundary.

According to the embodiment described above, when performing 180-degreeclipping of a VR image from flat display, a direction of an image end onthe flat display can be set to a center region after the clipping.

In addition, in a case where the VR image is not in a clipping editingstate, user operability can also be improved by feeding an image withleft and right scrolls.

It should be noted that the various controls described above as controlsto be performed by the CPU 101 may be carried out by one piece ofhardware or a plurality of pieces of hardware (for example, a pluralityof processors or circuits) may control an entire apparatus by sharingprocesses.

In addition, while the present invention has been described in detail onthe basis of a preferred embodiment thereof, it is to be understood thatthe present invention is not limited to the specific embodiment andvarious modes that does not constitute departures from the scope of theinvention are also included in the present invention. Furthermore, theembodiment described above simply represents an example of the presentinvention and the embodiment can also be combined with otherembodiments.

Moreover, while an example in which the present invention is applied toa smartphone has been described in the embodiment presented above, thepresent invention is not limited to this example and can be applied toany apparatus capable of displaying a VR image on a display unit. Forexample, the present invention can be applied to a personal computer, aPDA, a mobile phone terminal, a mobile image viewer, a printer apparatusequipped with a display, a digital photo frame, a music player, a gamedevice, an electronic book reader, a video player, and the like. Thepresent invention can also be applied to a digital camera, a televisionapparatus, a projection apparatus, a tablet terminal, an AI speaker, adomestic electrical appliance, a vehicle-mounted apparatus, medicalequipment, and the like.

When clipping a VR image with a narrower range from a VR image on flatdisplay, a region near a screen end of the flat display can bedesignated as a clipping center.

According to the embodiment described above, the followingconfigurations are disclosed. Specifically,

(A1) An electronic device including:

a display controlling unit configured to control so that flat displayinvolving fitting a first video range of a VR content into a rectangleis performed on a first screen and an indicator indicating a region tobe clipped from the VR content is displayed on the VR content beingdisplayed by flat display in a rectangular region on a second screen;

a switching unit configured to switch the first screen to the secondscreen in accordance with a setting instruction of a clipping range froma user;

a control unit configured to control so that

cyclical scroll display of a same content is performed on the secondscreen, the cyclical scroll display involving, in accordance with afirst operation, scrolling the VR content being displayed by flatdisplay in a first direction without scrolling the indicator and, inaccordance with the scrolling of the VR content, sequentiallydisplaying, in the first direction from an end in a second directionthat is an opposite direction to the first direction in the rectangularregion, an image region corresponding to a scroll amount in the firstdirection among the VR content, and that

the cyclical scroll display is not performed on the first screen evenwhen the first operation is performed: and

a generating unit configured to generate an edited VR content includinga second video range that is narrower than the first video range amongthe VR content on the basis of a region indicated by the indicator onthe second screen.

(A2) The electronic device according to (A1), wherein

the control unit causes a center of a range indicated by the indicatorin the VR content being displayed by flat display and a center of thesecond video range to coincide or approximately coincide with eachother.

(A3) The electronic device according to (A1) or (A2), wherein theindicator indicates a range that is narrower than the rectangularregion.

(A4) The electronic device according to any one of (A1) to (A3), wherein

the first direction is a rightward direction or a leftward directionrelative to the VR content.

(A5) The electronic device according to any one of (A1) to (A4), wherein

an orientation of the second video range changes in a case where thefirst video range is mapped onto a virtual sphere in accordance with thecyclical scroll display in the first direction on the second screen.

(A6) The electronic device according to any one of (A1) to (A5), wherein

the control unit does not perform a scroll in a third direction that isperpendicular to the first direction on the second screen.

(A7) The electronic device according to any one of (A1) to (A6), wherein

an elevation or a depression angle of the second video range is notchanged in a case where the first video range is mapped onto a virtualsphere in accordance with a scroll of the VR content on the secondscreen.

(A8) The electronic device according to any one of (A1) to (A7), wherein

the first operation is a slide operation involving touching therectangular region and moving the touch position in the first direction.

(A9) The electronic device according to any one of (A1) to (A8), wherein

when the setting instruction is accepted from the user, the control unitconfirms with the user as to whether or not zenith correction is to beperformed.

(A10) The electronic device according to any one of (A1) to (A9),wherein

the control unit accepts an instruction to perform clipping centered ona zenith or clipping centered on a nadir of the VR content by anoperation that does not involve a scroll operation.

(A11) The electronic device according to any one of (A1) to (A10)further including:

a recording unit configured to generate and record a file storing theedited VR content generated by the generating unit.

(A12) The electronic device according to (A11), wherein

the recording unit generates one still image file storing a plurality ofimages on the basis of the second video range.

(A13) The electronic device according to (A12), wherein

a character string “.vr” is described before an extension in a file nameof the still image file.

(A14) The electronic device according to any one of (A11) to (A13),wherein

the recording unit generates one moving image file in which a pluralityof moving images on the basis of the second video range are arranged.

(A15) The electronic device according to (A14), wherein

a character string “.vr” is described before an extension in a file nameof the moving image file.

(A16) A control method of an electronic device, including the steps of:

controlling so that flat display involving fitting a first video rangeof a VR content into a rectangle is performed on a first screen and anindicator indicating a region to be clipped from the VR content isdisplayed on the VR content being displayed by flat display in arectangular region on a second screen;

switching the first screen to the second screen in accordance with asetting instruction of a clipping range from a user:

controlling so that cyclical scroll display of a same content isperformed on the second screen, the cyclical scroll display involving,in accordance with a first operation, scrolling the VR content beingdisplayed by flat display in a first direction without scrolling theindicator and, in accordance with the scrolling of the VR content,sequentially displaying, in the first direction from an end in a seconddirection that is an opposite direction to the first direction in therectangular region, an image region corresponding to a scroll amount inthe first direction among the VR content, and that the cyclical scrolldisplay is not performed on the first screen even when the firstoperation is performed: and

generating an edited VR content including a second video range that isnarrower than the first video range among the VR content on the basis ofa region indicated by the indicator on the second screen.

(A17) A program which causes a computer to function as the respectiveunits of the electronic device according to any one of (A1) to (A15).

(A18) A computer-readable storage medium storing a program which causesa computer to function as the respective units of the electronic deviceaccording to any one of (A1) to (A15).

When clipping a 180-degree clipping region from a VR image, a user canpanoramically view the entire VR image by performing flat display inwhich a 360-degree valid video range is fitted into a rectangle.However, in case of designating a partial range such as a 180-degreeclipping region in the state of flat display, there is a problem in thatit is difficult to designate an end region on the flat display as aclipping center. In particular, there is a problem in that, in case ofdesiring to confirm a state of a screen end to be actually clipped, itis difficult to confirm the state prior to the clipping. Inconsideration of such problems, the embodiment described above alsodiscloses the following configurations so that a state of an end of apartial range displayed on a flat display of a VR image can be confirmedmore accurately. Specifically,

(B1) An electronic device including:

a display controlling unit configured to perform flat display involvingfitting a first video range of a VR content into a rectangle and whichdisplays an indicator indicating a region to be clipped from the VRcontent being subjected to the flat display; an accepting unitconfigured to accept, from a user, a confirmation instruction of aboundary of a second video range that is narrower than the first videorange on the basis of the region; and

a boundary display configured to control so that, in accordance with theconfirmation instruction, a range including the boundary of the secondvideo range among the omnidirectional VR content is displayed in asecond display mode having been deformed from the flat display in such amanner that the boundary can be identified.

(B2) The electronic device according to (B1), wherein

the second display mode is a display mode involving displaying bymapping on a virtual sphere.

(B3) The electronic device according to (B1) or (B2), wherein

the accepting unit accepts a clipping instruction to clip the secondvideo range, and

the electronic device further includes a generating unit configured togenerate, in accordance with the clipping instruction, an edited VRcontent including the second video range.

(B4) The electronic device according to (B3), further including arecording unit configured to generate and record a file storing theedited VR content generated by the generating unit.

(B5) The electronic device according to (B4), wherein

the recording unit generates one still image file storing a plurality ofimages on the basis of the second video range.

(B6) The electronic device according to (B5), wherein

a character string “.vr” is described before an extension in a file nameof the still image file.

(B7) The electronic device according to any one of (B4) to (B6), wherein

the recording unit generates one moving image file in which a pluralityof moving images on the basis of the second video range are arranged.

(B8) The electronic device according to (B7), wherein

a character string “.vr” is described before an extension in a file nameof the moving image file.

(B9) The electronic device according to any one of (B1) to (B8), wherein

the confirmation instruction is an operation by a user of designating aboundary to be a confirmation object in the region enclosed by theindicator.

(B10) The electronic device according to any one of (B1) to (B9),wherein

the boundary display unit displays a range which includes a boundary ofthe second video range among the VR content and which is narrower thanthe second video range.

(B11) A control method of an electronic device, including the steps of:

performing flat display involving fitting a first video range of a VRcontent into a rectangle and displaying an indicator indicating a regionto be clipped from the VR content being subjected to the flat display;

accepting, from a user, a confirmation instruction of a boundary of asecond video range that is narrower than the first video range on thebasis of the region: and

controlling so that, in accordance with the confirmation instruction, arange including the boundary of the second video range among theomnidirectional VR content is displayed in a second display mode havingbeen deformed from the flat display in such a manner that the boundarycan be identified.

(B12) A program which causes a computer to function as the respectiveunits of the electronic device according to any one of (B1) to (B10).

(B13) A computer-readable storage medium storing a program which causesa computer to function as the respective units of the electronic deviceaccording to any one of (B1) to (B10).

<Other Embodiments>

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2018-248299, filed on Dec. 28, 2018, Japanese Patent Application No.2018-248283, filed on Dec. 28, 2018, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An electronic device comprising: at least onememory and at least one processor which function as: a displaycontrolling unit configured to control so that flat display involvingfitting a first video range of a VR content into a rectangle isperformed on a first screen and an indicator indicating a region to beclipped from the VR content is displayed on the VR content beingdisplayed by flat display in a rectangular region on a second screen; aswitching unit configured to switch the first screen to the secondscreen in accordance with a setting instruction of a clipping range froma user; a control unit configured to control so that cyclical scrolldisplay of a same content is performed on the second screen, thecyclical scroll display involving, in accordance with a first operation,scrolling the VR content being displayed by flat display in a firstdirection without scrolling the indicator and, in accordance with thescrolling of the VR content, sequentially displaying, in the firstdirection from an end in a second direction that is an oppositedirection to the first direction in the rectangular region, an imageregion corresponding to a scroll amount in the first direction among theVR content, and that the cyclical scroll display is not performed on thefirst screen even when the first operation is performed; and agenerating unit configured to generate an edited VR content including asecond video range that is narrower than the first video range among theVR content on the basis of a region indicated by the indicator on thesecond screen.
 2. The electronic device according to claim 1, whereinthe control unit causes a center of a range indicated by the indicatorin the VR content being displayed by flat display and a center of thesecond video range to coincide or approximately coincide with eachother.
 3. The electronic device according to claim 1, wherein theindicator indicates a range that is narrower than the rectangularregion.
 4. The electronic device according to claim 1, wherein the firstdirection is a rightward direction or a leftward direction relative tothe VR content.
 5. The electronic device according to claim 1, whereinan orientation of the second video range changes in a case where thefirst video range is mapped onto a virtual sphere in accordance with thecyclical scroll display in the first direction on the second screen. 6.The electronic device according to claim 1, wherein the control unitdoes not perform a scroll in a third direction that is perpendicular tothe first direction on the second screen.
 7. The electronic deviceaccording to claim 1, wherein an elevation or a depression angle of thesecond video range is not changed in a case where the first video rangeis mapped onto a virtual sphere in accordance with a scroll of the VRcontent on the second screen.
 8. The electronic device according toclaim 1, wherein the first operation is a slide operation involvingtouching the rectangular region and moving the touch position in thefirst direction.
 9. The electronic device according to claim 1, whereinin a case where the setting instruction is accepted from the user, thecontrol unit confirms with the user as to whether or not zenithcorrection is to be performed.
 10. The electronic device according toclaim 1, wherein the control unit accepts an instruction to performclipping centered on a zenith or clipping centered on a nadir of the VRcontent by an operation that does not involve a scroll operation. 11.The electronic device according to claim 1, wherein the at least onememory and at least one processor further function as: a recording unitconfigured to generate and record a file storing the edited VR contentgenerated by the generating unit.
 12. The electronic device according toclaim 11, wherein the recording unit generates one still image filestoring a plurality of images on the basis of the second video range.13. The electronic device according to claim 12, wherein a characterstring “.vr” is described before an extension in a file name of thestill image file.
 14. The electronic device according to claim 11,wherein the recording unit generates one moving image file in which aplurality of moving images on the basis of the second video range arearranged.
 15. The electronic device according to claim 14, wherein acharacter string “.vr” is described before an extension in a file nameof the moving image file.
 16. A control method of an electronic device,comprising the steps of: controlling so that flat display involvingfitting a first video range of a VR content into a rectangle isperformed on a first screen and an indicator indicating a region to beclipped from the VR content is displayed on the VR content beingdisplayed by flat display in a rectangular region on a second screen;switching the first screen to the second screen in accordance with asetting instruction of a clipping range from a user; controlling so thatcyclical scroll display of a same content is performed on the secondscreen, the cyclical scroll display involving, in accordance with afirst operation, scrolling the VR content being displayed by flatdisplay in a first direction without scrolling the indicator and, inaccordance with the scrolling of the VR content, sequentiallydisplaying, in the first direction from an end in a second directionthat is an opposite direction to the first direction in the rectangularregion, an image region corresponding to a scroll amount in the firstdirection among the VR content, and that the cyclical scroll display isnot performed on the first screen even when the first operation isperformed; and generating an edited VR content including a second videorange that is narrower than the first video range among the VR contenton the basis of a region indicated by the indicator on the secondscreen.
 17. A non-transitory computer readable medium that stores aprogram, wherein the program causes a computer to execute the steps of:controlling so that flat display involving fitting a first video rangeof a VR content into a rectangle is performed on a first screen and anindicator indicating a region to be clipped from the VR content isdisplayed on the VR content being displayed by flat display in arectangular region on a second screen; switching the first screen to thesecond screen in accordance with a setting instruction of a clippingrange from a user; controlling so that cyclical scroll display of a samecontent is performed on the second screen, the cyclical scroll displayinvolving, in accordance with a first operation, scrolling the VRcontent being displayed by flat display in a first direction withoutscrolling the indicator and, in accordance with the scrolling of the VRcontent, sequentially displaying, in the first direction from an end ina second direction that is an opposite direction to the first directionin the rectangular region, an image region corresponding to a scrollamount in the first direction among the VR content, and that thecyclical scroll display is not performed on the first screen even whenthe first operation is performed; and generating an edited VR contentincluding a second video range that is narrower than the first videorange among the VR content on the basis of a region indicated by theindicator on the second screen.